Skip to content

P2P WebRTC transport + daemon: seamless CDP over data channel#15

Open
iWedmak wants to merge 17 commits into
masterfrom
feature/daemon
Open

P2P WebRTC transport + daemon: seamless CDP over data channel#15
iWedmak wants to merge 17 commits into
masterfrom
feature/daemon

Conversation

@iWedmak

@iWedmak iWedmak commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Python SDK: P2P WebRTC transport + daemon + seamless lifecycle

What's included:

  1. P2P WebRTC transport — aiortc-based RTCPeerConnection with ceki-cmd data channel for CDP commands/responses. Primary CDP path with WS fallback. 41 tests.

  2. Seamless integration — SDK API unchanged (connect → rent → cdp/navigate/screenshot). P2P is under the hood, user code stays the same.

  3. DC→WS fallback — if P2P data channel send fails mid-session, auto-fallback to WS without losing the command. _p2p_fallback flag prevents repeated DC attempts.

  4. P2P lifecycle monitoring — connection state + data channel state callbacks with full logging. WebRTC failure → cleanup _p2p → transparent WS fallback.

  5. Screenshot race fix — per-command transport tagging: skip WS echo for DC-sent commands. Prevents empty screenshot from WS echo racing ahead of full DC response.

  6. Orphan cleanup — SIGINT/SIGTERM signal handlers in connect() shut down all browsers + disconnect. Client + Browser context managers (async with) for clean teardown.

  7. ceki-daemon — persistent renter-process with local HTTP/JSON IPC server. Auto-starts on rent command.

  8. CEKI_FORCE_WS — env var to force WS path for regression testing.

Usage:

from ceki_sdk import connect, ConnectOptions

async with connect(api_key) as client:
    async with await client.rent(schedule_id) as browser:
        result = await browser.navigate('https://example.com')
        data = await browser.screenshot(format='png')

No P2P-specific code needed — it's automatic.

ceki-plugin added 10 commits July 19, 2026 15:09
…e import logging, visible stderr error

- Sync __init__.__version__ to 2.36.0 matching pyproject.toml
- Close log_file handle after subprocess.Popen in _ensure_daemon() and _cmd_daemon_start()
- Move import logging to top-level imports
- Add user-visible stderr error on daemon auto-start failure in _ensure_daemon()
- Remove redundant is_running() check after for-loop (proc.poll() already covers it)
- Add .claude/ .graphifyignore .mypy_cache/ to .gitignore
Missing [project.scripts] ceki-daemon = ceki_sdk.daemon:main entry
point — the published PyPI package had no ceki-daemon binary.
Every Joe review round (4400, 4439, 4444) cited this as the blocker.
Same wire call, same args, different name — lets AI agents express
'edit the task' intent instead of 'propose a correction'.

- New top-level subcommand 'edit' (ceki edit <eid> --label ...)
- Delegates to ContractClient.propose() under the hood
- Accepts all the same args: --status, --label, --desc, --start,
  --end, --date, --duration, --amount, --currency, --benefitable, --tags
Added 'edit' as a contract subparser (ceki contract edit <eid> ...)
that delegates to ContractClient.propose() — same args, same wire
call, different name for 'edit task' intent.
- WebRTCTransport: wraps aiortc RTCPeerConnection for P2P CDP
  over DataChannel('ceki-cmd') with ICE candidate queuing
- _client.py: P2P init, webrtc.offer/answer/ice_candidate dispatch
- _browser.py: DC-aware Browser.send() — P2P transport preferred
  over WS fallback when ceki-cmd DC is open
- CEKI_FORCE_WS env flag, CEKI_TURN_SERVERS env,
  CEKI_ICE_TRANSPORT_POLICY env
- 41 unit tests in tests/test_webrtc_p2p.py (134/135 pass,
  1 pre-existing failure in test_cli.py)
- pyproject.toml: aiortc>=1.9,<2 dependency
BUG: In P2P mode (cmd_dc_open=true), CDP responses arrive via BOTH
WS relay (as cdp_response) and the ceki-cmd data channel. The WS
response arrives first but has empty result {} for large payloads
like screenshot base64. The pending CDP future resolves with empty
data before the DC response with real data arrives.

FIX: In _reader_loop, when P2P is active and cmd_dc_open is true,
skip WS cdp_response messages. The DC response arrives and resolves
the future with complete data. When DC is not open (WS-only mode),
WS cdp_response is handled as before.

Affected: screenshot(), snapshot() return 0 bytes in P2P mode without this fix.
v1 (25cb42e) was too broad: skipped ALL WS cdp_response when P2P
DC was open, including responses for commands sent via WS before
P2P connected.

v2 fix: tag each pending CDP future with its transport ('dc' or 'ws')
at send time. _on_cdp_response only skips WS echo for futures tagged
as 'dc' (the screenshot race). WS-sent commands resolve normally even
if P2P connects before their response arrives.

Key changes:
- Browser.send(): tag fut._cdp_transport before storing
- Browser._on_cdp_response(): check tag + message origin;
  skip WS echo for DC-sent commands, accept all others
@iWedmak iWedmak changed the title feat: ceki-daemon persistent renter-process + CLI IPC P2P WebRTC transport + daemon: seamless CDP over data channel Jul 24, 2026
ceki-plugin added 7 commits July 24, 2026 08:32
…context managers, orphan cleanup

- Browser.send(): catch DC send failure → auto-fallback to WS with _p2p_fallback flag
- Client._init_p2p(): wire on_connection_state + on_data_channel_state for lifecycle logging
- Client._p2p_set_remote(): handle webrtc.answer with error logging
- Client.__aenter__/__aexit__: context manager for async with
- Browser.__aenter__/__aexit__: context manager for async with
- connect(): register SIGINT/SIGTERM cleanup handlers for orphan prevention
- Add _dc_open_event (asyncio.Event) to WebRTCTransport
- Clear before createOffer, set when ceki-cmd DC opens
- Browser.send() now waits up to 5s for DC readiness before sending CDP
- Timeout → WS fallback (sticky _p2p_fallback for the session)
- Prevents startup-race WS congestion that starves heartbeat ping

Part of ev 4864 P2P seamless review fix (GAP1/Joe)
join() sent {type:'attach', browser_id} but relay attachSchema
expects schedule_id. Result: -1014 Invalid attach payload.
Fix: send schedule_id matching relay schema.
_dispatch() didn't handle webrtc.answer — P2P answer never reached
WebRTCTransport.setRemoteDescription. Add dispatch to _p2p_set_remote.
Also add timing debug logging in Browser.send() and fix CDP cleanup.
…ntee first CDP via DC (ev 4902)

- _p2p_ready Event signals when self._p2p is assigned
- rent() waits up to 15s for P2P transport init before returning Browser
- Browser.send() DC timeout 5s→30s, no permanent _p2p_fallback on TimeoutError
- ConnectionError/OSError still sets permanent WS fallback (DC broken)
wait_dc_open() was placed before create_offer() — DC didn't exist
yet, resulting in 15s timeout and permanent P2P disable.

Fix: wait for DC open AFTER offer is created and sent. _p2p_ready
signals only when DC is actually usable. rent() then returns Browser
with P2P already active — first CDP guaranteed via DC.

Also await _p2p_set_remote in _dispatch() instead of create_task
for proper ordering.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant